The present invention generally relates to enhanced vision systems and, more particularly, to situational awareness components of an enhanced vision system and a method for enhancing flight crew situational awareness.
The maintenance of situational awareness has always been and is still today a pilot's uppermost concern. Loss of situational awareness is most often a main factor in airplane accidents, which is true for commercial aircraft as well as for military aircraft. One of the most dangerous challenges military aviators face is poor visibility, especially during operations in unprepared environments. In recent years, enhanced vision systems have been developed that improve the ability of pilots to see airport features and surrounding terrain at night and during periods of reduced visibility while flying close to the ground. Still, with constantly increasing complexity of the tasks of a flight crew, especially of a crew flying a military plane involved in war actions, there is a need to further develop existing enhanced vision systems in order to enhance flight crew situational awareness. For example, a flight crew of a military airplane carrying a weapon, such as the advanced tactical laser (ATL) or a gunship, needs to maintain common awareness for battle space while focused on individual tasks. A weapon operator may be narrowly focused on a single target while cockpit crewmembers may have broad situational awareness. If the cockpit crew becomes aware of an important new development, the cockpit crew needs to rapidly cue the weapon operator. Similarly, a battle manager may observe an emerging threat to the aircraft and need to notify the cockpit crew immediately, for example, by visually cueing them. Furthermore, it may happen that only one of the crewmembers becomes aware of a newly developing situation. In this case, this crewmember needs to be able to communicate with the other crewmembers. Presently, there is no effective solution on how to share visual information among flight crewmembers, such as a weapon operator or a battle manager, to rapidly cue other members to targets or situations observed by one crewmember.
Furthermore, a weapon operator, for example, an ATL weapon operator, needs to maintain situational awareness of multiple targets, while focused on a single target at a time. Currently, the weapon operator is required to search for each target separately. While focused on one target, other targets may move under cover, making it harder for the weapon operator to search for them. Therefore, if more than one target needs to be tracked at the same time, prior art requires one operator for each target to be tracked.
Another task that requires situational awareness of the flight crewmembers, for example, of a low flying helicopter, is obstacle detection, for example, of power lines, since striking a power line will be disastrous for any aircraft. Passive power line detection systems for aircraft have been developed, for example, U.S. Patent Application No. US2002/0153485 A1 published by Nixon et. al. This obstacle detection system determines the presence of small, curvilinear objects, such as power lines. While the detected objects will be displayed for the pilot such that evasive maneuvers can be performed by the pilot as necessary, the pilot cannot change his line of sight and look to the right or left. The pilot's line of sight needs to be where he suspects, for example, power lines.
Prior artificial vision systems typically use a single turreted sensor system that is slaved to a pilot's line of sight. As the pilot turns his head, the entire turret rotates to follow his line of sight. Consequently, all users can see imagery only in the pilot's line of sight. Thus, if the pilot is looking to one side, an operator aide, such as an obstacle detection system or a tracking system, can only view imagery in that direction. Using prior art vision systems it is not possible that the users can independently monitor views in different directions.
Currently a next generation of enhanced vision systems (EVS) is being developed, for example, the enhanced vision system described by Yelton, Bernier, and Sanders-Reed in Proc SPIE, 5424, April 2004, hereby incorporated by reference, combine imagery from multiple sensors, possibly running at different frame rates and pixel counts, onto a display. In the case of a helmet mounted display (HDM), the user line of sight is continuously changing with the result that the sensor pixels rendered on the display are changing in real time. In a prior art enhanced vision system, the various sensors provide overlapping fields of view, which requires stitching imagery together to provide a seamless mosaic to the user. Furthermore, different modality sensors may be present requiring the fusion of imagery from the sensors having a common field of view. Still further, it is possible to combine sensor imagery with synthetic imagery, such as 3D terrain from digital elevation maps, overhead satellite imagery, or flight path symbology. The output of an enhanced vision system may be presented on a head-down, head-up, or helmet mounted display. All of this takes place in a dynamic flight environment where the aircraft (with fixed mounted sensors) is changing position and orientation while the users are independently changing their lines of sight. Modern enhanced vision systems, for example, the enhanced vision system described by Yelton, Bernier, and Sanders-Reed, may provide new opportunities for visual sharing information and for using independent operator aides and intelligent agents, not available in systems pre-dating enhanced vision systems. However, current prior art enhanced vision systems are “dumb” systems in the sense that these systems provide integrated imagery to a human user who supplies all the intelligence for interpretation.
Prior art further includes, for example, an enhanced vision system called “Flying Infrared for Low-level Operations” (FLILO) disclosed by Guell in IEEE AES Systems Magazine, September 2000, pp. 31-35. FLILO enhances situational awareness for safe low level/night time and moderate weather flight operations, such as take-off, landing, taxiing, approaches, drop zone identification and short austere airfield operations. FLILO provides electronic/real time vision to the pilots through a series of imaging sensors, an image processor, and a wide field-of-view see-through helmet mounted display integrated with a head tracker. While enhancing the situational awareness of a flight crew, the FLILO enhanced vision system does not offer visual communication between flight crew members or allows the user to maintain situational awareness of multiple tasks.
As can be seen, there is a need for increasing not only the pilot's situational awareness but also the flight crew's situational awareness. Furthermore, there is a need to enable instant visual communication between a cockpit crew, a weapon operator, a battle manager, or other personnel of a military aircraft. Also, there is a need to share visual information among cockpit crewmembers. Moreover, a need exists to enable a weapon operator to maintain situational awareness of multiple targets. Still further, there is a need to add “intelligent” components to existing enhanced vision systems that may provide help to human users or that may replace human users.
There has, therefore, arisen a need to provide components that enable sharing of visual information among flight crew members and that may be added to existing enhanced vision systems. There has further arisen a need to provide components that reduce the workload of an operator, such as a weapon operator, and that may be added to prior art enhanced vision systems. There has still further arisen a need to provide “intelligent” components that may handle various flight and battle operations based on the broad area coverage provided by an enhanced vision system.